Object detecting apparatus and object detecting method

ABSTRACT

An object of the present invention is to further improve the detection accuracy of a lateral position of a target in an object detecting apparatus for detecting an object by using a radar and a monocular image sensor. In the present invention, a target corresponding to a target recognized by the radar is extracted and a right edge and a left edge of the target are acquired from an image picked up by the monocular image sensor. Further, locus approximation lines, which are straight lines or predetermined curved lines for approximating loci of the right edge and the left edge, are derived for the both edges. The edge, which has a larger number of edges existing on the locus approximation line, is selected as a true edge of the target from the right edge and the left edge. The lateral position of the target is derived on the basis of the position of the selected edge.

TECHNICAL FIELD

The present invention relates to an object detecting apparatus and anobject detecting method in which an object is detected on the basis ofinformation acquired by a radar and an image sensor.

BACKGROUND ART

In recent years, safety systems such as PCS (Pre-crash safety system)and the like are developed in order to avoid the collision or reduce thedamage caused by the collision. In order to preferably realize such asafety system, it is necessary to correctly grasp, for example, theposition and the size of an obstacle including, for example, apedestrian and a vehicle other than a subject vehicle as well as thedistance between the obstacle and the subject vehicle. An objectdetecting apparatus, which is based on the use of a radar and a stereoimage sensor, is known as a technique for grasping, for example, theposition, the size, and the distance as described above.

When the radar is used, it is possible to recognize a target (objectprovided as a detection objective) as a point of reflection of theelectromagnetic wave. Accordingly, it is possible to acquire or obtainthe position of the target. However, it is difficult to correctlyacquire the edge of the target by means of the radar. On the other hand,it is possible to highly accurately acquire the edge of the target froman image picked up or photographed by the stereo image sensor. Thus, theobject detecting apparatus as described above causes the fusion of thetarget information acquired by the radar and the target informationacquired from the image picked up by the stereo image sensor.Accordingly, it is possible to improve the object detecting ability ofthe object detecting apparatus.

However, when the stereo image sensor is used as the image sensor, it isnecessary to secure a relatively large space in order to install thestereo image sensor. Further, the cost is relatively expensive as wellin order to realize the object detecting apparatus. Therefore, it isdemanded that an object detecting apparatus, which has the function andthe performance equivalent to those obtained when the stereo imagesensor is used, is realized by using a monocular image sensor in placeof the stereo image sensor.

Patent Document 1 discloses a vehicle obstacle recognizing apparatuswhich uses a millimeter wave radar and a monocular camera. The vehicleobstacle recognizing apparatus comprises an object informationcalculating means, an image processing means, and a vehicle informationacquiring means. The object information calculating means calculates theobject information including, for example, the relative lateral positionand the relative distance with respect to the detection object from theoutput of the millimeter wave radar. The image processing meansprocesses an image picked up or photographed by the monocular camera onthe basis of the calculation result obtained by the object informationcalculating means. The vehicle obstacle recognizing apparatus judges thepossibility of the detection object to behave as an obstacle on thebasis of the outputs of at least the object information calculatingmeans and the vehicle information acquiring means. Further, it is judgedwhether or not the output of the image processing means is effective tojudge the obstacle, on the basis of the calculation result obtained bythe object information calculating means. Only when the output of theimage processing means is effective, the output of the image processingmeans is also used to judge the obstacle.

Patent Document 2 discloses an object detecting apparatus for acquiringimage information and distance information from a camera and a radar. Inthe object detecting apparatus, the direction vector of the edge, thedirection vector variance of the edge, the edge intensity, and the edgeintensity variance are calculated from the image information. The typeof an objective is judged on the basis of at least one of them and thedistance with respect to the detection objective.

PRIOR ART DOCUMENTS Patent Documents

-   Patent Document 1: Japanese Patent Application Laid-Open No.    2008-276689-   Patent Document 2: Japanese Patent Application Laid-Open No.    2007-288460

DISCLOSURE OF THE INVENTION Problem to be Solved by the Invention

When the monocular image sensor is used as the image sensor in theobject detecting apparatus, it is possible to contemplate the spacesaving and the low cost. However, it is difficult to acquire any correctinformation in the distal direction or depth direction from the imagepicked up by the monocular image sensor. Therefore, when the left andright edges of a target are detected from an image picked up by themonocular image sensor, an edge of an object or a pattern, whichactually exists distally as compared with the target as viewed from thesubject vehicle, is erroneously detected as the edge of the target insome cases. If the position in the lateral direction of the target(lateral position) is derived on the basis of the edge erroneouslydetected as described above, it is feared that the lateral position ofthe target may be erroneously detected as well.

The present invention has been made taking the foregoing problem intoconsideration, an object of which is to provide a technique which makesit possible to further improve the detection accuracy of the lateralposition of a target in an object detecting apparatus for detecting anobject on the basis of target information acquired by a radar and targetinformation acquired from an image picked up by a monocular imagesensor.

Means for Solving the Problem

In the present invention, a right edge and a left edge of a target areacquired or obtained from an image picked up or photographed by amonocular image sensor. Further, locus approximation lines, which arestraight lines or predetermined curved lines for approximating loci ofthe right edge and the left edge, are derived for the both edges. Theedge, which is included in the right edge and the left edge and whichhas a larger number of edges existing on the locus approximation line,is selected as the true edge of the target. The lateral position of thetarget is derived on the basis of the position of the selected edge.

More specifically, the object detecting apparatus according to a firstinvention resides in an object detecting apparatus for detecting anobject on the basis of target information acquired by a radar and targetinformation acquired from an image picked up by a monocular imagesensor, the object detecting apparatus comprising:

an edge acquiring means which extracts a target corresponding to atarget recognized by the radar, from the image picked up by themonocular image sensor and which acquires a right edge and a left edgeof the extracted target;

a locus approximation line deriving means which derives, for the bothedges, locus approximation lines as straight lines or predeterminedcurved lines for approximating loci of the right edge and the left edgeacquired by the edge acquiring means;

a selecting means which selects, as a true edge of the target, the edgehaving a larger number of edges existing on the locus approximationline, from the right edge and the left edge acquired by the edgeacquiring means; and

a lateral position deriving means which derives a lateral position ofthe target on the basis of a position of the edge selected as the trueedge by the selecting means.

According to the present invention, the edge which has the higherreliability and which is selected from the right edge and the left edgeof the target acquired from the image picked up by the monocular imagesensor, is selected as the true edge of the target. The lateral positionof the target is derived on the basis of the position of the edge havingthe higher reliability. Therefore, it is possible to further improve thedetection accuracy of the lateral position of the target.

The object detecting apparatus according to the present invention mayfurther comprise a weight applying means which applies reliabilityweights to the right edge and the left edge acquired by the edgeacquiring means. In this case, the weight applying means applies theweights to the right edge and the left edge so that the edge, which isdisposed nearer to a position of the target recognized by the radar, hasa higher reliability. Further, the object detecting apparatus accordingto the present invention may comprise a reliability total valuecalculating means which totalizes a plurality of reliabilities havingthe weights applied by the weight applying means for each of the rightedge and the left edge to thereby calculate total values of thereliabilities for the both edges.

In the present invention, the number of the edges existing on the locusapproximation line is identical between the right edge and the left edgein some cases. In such a situation, the selecting means may select, asthe true edge of the target, the edge which has the larger total valueof the reliabilities calculated by the reliability total valuecalculating means, from the right edge and the left edge.

Accordingly, the edge, which has the higher reliability and which isselected from the right edge and the left edge of the target, can bealso selected as the true edge of the target.

In the present invention, the lateral position deriving means mayinclude a locus predicting means and a collision position predictingmeans. The locus predicting means predicts a future locus of the edgeselected as the true edge by the selecting means. The collision positionpredicting means predicts a collision position between the target and avehicle as a position at which a distance between the edge and thevehicle in the front-back direction is zero on the basis of the futurelocus of the edge predicted by the locus predicting means. In this case,a lateral position of a center in a lateral direction of the target(hereinafter referred to as “target center”), which is to be provided atthe collision position, may be derived by the lateral position derivingmeans on the basis of a position of the edge selected as the true edgeby the selecting means at the collision position predicted by thecollision position predicting means.

Accordingly, it is possible to detect the lateral position of the targetcenter at the collision position between the target and the vehicle.

In the object detecting apparatus according to the present invention,the lateral position deriving means may include a lateral widthestimating means which estimates a lateral width of the target. In thiscase, a position, which is deviated toward the other edge by ½ of thelateral width of the target estimated by the lateral width estimatingmeans from the position of the edge selected as the true edge by theselecting means, may be derived by the lateral position deriving meansas a lateral position of the target center.

Accordingly, it is possible to detect the lateral position of the targetcenter highly accurately.

The object detecting method according to a second invention resides inan object detecting method for detecting an object on the basis oftarget information acquired by a radar and target information acquiredfrom an image picked up by a monocular image sensor, the objectdetecting method comprising:

an edge acquiring step of extracting a target corresponding to a targetrecognized by the radar, from the image picked up by the monocular imagesensor and acquiring a right edge and a left edge of the extractedtarget;

a locus approximation line deriving step of deriving, for the bothedges, locus approximation lines as straight lines or predeterminedcurved lines for approximating loci of the right edge and the left edgeacquired in the edge acquiring step;

a selecting step of selecting, as a true edge of the target, the edgehaving a larger number of edges existing on the locus approximationline, from the right edge and the left edge acquired in the edgeacquiring step; and

a lateral position deriving step of deriving a lateral position of thetarget on the basis of a position of the edge selected as the true edgein the selecting step.

In the present invention, the lateral position of the target is alsoderived on the basis of the position of the edge having the higherreliability. Therefore, it is possible to further improve the detectionaccuracy of the lateral position of the target.

The object detecting method according to the present invention mayfurther comprise a weight applying step of applying reliability weightsto the right edge and the left edge acquired in the edge acquiring step.In this case, the weights are applied to the right edge and the leftedge in the weight applying step so that the edge, which is disposednearer to a position of the target recognized by the radar, has a higherreliability. Further, the object detecting method according to thepresent invention may comprise a reliability total value calculatingstep of totalizing a plurality of reliabilities having the weightsapplied in the weight applying step for each of the right edge and theleft edge to thereby calculate total values of the reliabilities for theboth edges.

In the present invention, the edge, which has the larger total value ofthe reliabilities calculated in the reliability total value calculatingstep, may be selected as the true edge of the target in the selectingstep, if the number of the edges existing on the locus approximationline is identical between the right edge and the left edge.

Accordingly, the edge, which has the higher reliability and which isselected from the right edge and the left edge of the target, can beselected as the true edge of the target.

In the first and second inventions, the “edge existing on the locusapproximation line” may include not only the edge existing at theposition completely coincident with the locus approximation line butalso the edge positioned within a predetermined allowable range from thelocus approximation line.

Advantageous Effect of the Invention

According to the present invention, it is possible to further improvethe detection accuracy of the lateral position of the target in theobject detecting apparatus for detecting the object on the basis of thetarget information acquired by the radar and the target informationacquired from the image picked up by the monocular image sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram illustrating a schematic arrangement of acollision predicting apparatus according to a first embodiment.

FIG. 2 shows block diagrams illustrating a schematic arrangement of anobject detecting section according to the first embodiment.

FIG. 3 shows a millimeter wave detection position and monocular imagedetection edges in relation to an image picked up by a monocular imagesensor.

FIG. 4 shows a detection result obtained when an obstacle is detected byusing a millimeter wave radar and a stereo image sensor.

FIG. 5 shows a detection result obtained when an obstacle is detected byusing a millimeter wave radar and the monocular image sensor.

FIG. 6 shows locus approximation lines of right edges and left edgesderived for monocular image detection edges according to the firstembodiment.

FIG. 7 shows the relationship among the locus approximation line of theselected edge, the predicted future locus, the collision position, andthe lateral position of the target center according to the firstembodiment.

FIG. 8 shows a flow chart illustrating a part of a collision judgmentflow according to the first embodiment.

FIG. 9 shows a flow chart illustrating a part of the collision judgmentflow according to the first embodiment.

FIG. 10 shows a block diagram illustrating a schematic arrangement of anobject detecting section according to a second embodiment.

FIG. 11 shows a detection result obtained when an obstacle is detectedby using a millimeter wave radar and a monocular image sensor.

FIG. 12 shows locus approximation lines of right edges and left edgesderived for monocular image detection edges according to the secondembodiment.

FIG. 13 shows the relationship among the locus approximation line of theselected edge, the predicted future locus, the collision position, andthe lateral position of the target center according to the secondembodiment.

FIG. 14 shows a flow chart illustrating a part of a collision judgmentflow according to the second embodiment.

THE MODE FOR CARRYING OUT THE INVENTION

Specified embodiments of the present invention will be explained belowon the basis of the drawings. For example, the size, the material, theshape, and the relative arrangement of any constitutive part describedin the embodiment of the present invention are not aimed to limit thetechnical scope of the invention only thereto unless specificallystated.

First Embodiment

A first embodiment of the present invention will be explained on thebasis of FIGS. 1 to 9.

(Schematic Arrangement)

In this section, an explanation will be made about a case in which thepresent invention is applied to a collision predicting apparatus. FIGS.1 and 2 show block diagrams illustrating a schematic arrangement of thecollision predicting apparatus according to this embodiment. Thecollision predicting apparatus 200 is carried on a vehicle 100, which isan apparatus to predict the collision between a subject vehicle 100 andan obstacle including, for example, another vehicle (object vehicle) anda pedestrian or walker. A warning apparatus 8 and a collisionpreventing/collision damage reducing system 9, which are operated if thecollision with the obstacle is predicted, are carried on the vehicle 100in addition to the collision predicting apparatus 200.

The collision predicting apparatus 200 comprises a millimeter wave radar1, a monocular image sensor 2, a steering angle sensor 3, a yaw ratesensor 4, a wheel pulse sensor 5, and ECU 10. The millimeter wave radar1 is attached to a front central portion of the vehicle 100. Themillimeter wave radar 1 performs the scanning in the horizontaldirection with an electromagnetic wave in a millimeter wave band forthose disposed in the frontward direction and the oblique frontwarddirection of the vehicle 100. Further, the millimeter wave radar 1receives the electromagnetic wave reflected by a surface of an objectdisposed outside the vehicle. Accordingly, the millimeter wave radar 1recognizes a target as a point of reflection of the electromagneticwave. The target information (for example, the relative position of thetarget with respect to the subject vehicle 100), which is acquired orobtained from the transmitted/received data of the millimeter wave, isinputted into ECU 10.

The monocular image sensor 2 is attached to a front central portion ofthe vehicle 100. The monocular image sensor 2 photographs or picks upthe image in the frontward direction and the oblique frontward directionof the vehicle 100. The picked up image is inputted as an image signalinto ECU 10.

The steering angle sensor 3 is attached, for example, to a steering rodof the vehicle 100. The steering angle sensor 3 detects the steeringangle of the steering wheel operated by a driver. The yaw rate sensor 4is provided at a central position of the body of the vehicle 100. Theyaw rate sensor 4 detects the yaw rate applied to the body. The wheelpulse sensor 5 is attached to a wheel portion of the vehicle 100. Thewheel pulse sensor 5 detects the wheel velocity of the vehicle. Outputsignals of the sensors are inputted into ECU 10.

ECU 10 has an object detecting section 6 and a collision judging section7. The object detecting section detects the obstacle on the basis of thetarget information acquired by the millimeter wave radar 1 and thetarget information acquired from the image picked up by the monocularimage sensor 2. The collision judging section 7 judges whether or notthe obstacle detected by the object detecting section 6 and the subjectvehicle 100 collide with each other. Details will be described later onabout the object detecting method performed in the object detectingsection 6 and the collision judging method performed in the collisionjudging section 7.

FIG. 2 shows block diagrams illustrating a schematic arrangement of theobject detecting section 6 according to this embodiment. As shown inFIG. 2( a), the object detecting section 6 has an edge detecting section61, a locus approximation line deriving section 62, a selecting section63, and a lateral position deriving section 64. As shown in FIG. 2( b),the lateral position deriving section 64 has a locus predicting section641, a collision position predicting section 642, a lateral widthestimating section 643, and a target center lateral position derivingsection 644. Details of the respective sections 61 to 64 and 641 to 644will be described later on.

Further, ECU 10 has calculating sections for calculating variousparameters required to judge the collision in the collision judgingsection including, for example, an estimated curve radius calculatingsection, a subject vehicle velocity calculating section, a subjectvehicle orbit calculating section, an obstacle velocity calculatingsection, and an obstacle movement distance calculating section (notshown). For example, the estimated curve radius calculating sectioncalculates the estimated curve radius of the subject vehicle 100 on thebasis of the steering angle signal inputted from the steering anglesensor 3 and the yaw rate signal inputted from the yaw rate sensor 4.The subject vehicle velocity calculating section calculates the vehiclevelocity of the subject vehicle 100 on the basis of the wheel velocitysignal inputted from the wheel pulse sensor 5. The subject vehicle orbitcalculating section calculates the orbit of the subject vehicle 100 onthe basis of the estimated curve radius signal inputted from theestimated curve radius calculating section. The obstacle velocitycalculating section calculates the movement velocity of the obstacledetected by the object detecting section 6 on the basis of the targetinformation. The obstacle movement distance calculating sectioncalculates the movement distance after the detection of the obstacledetected by the object detecting section 6 on the basis of the targetinformation.

If it is judged by the collision judging section 7 that the subjectvehicle will collide with the obstacle, the ON signal is transmittedfrom ECU 10 to the warning apparatus 8 and the collisionpreventing/collision damage reducing system 9. When the warningapparatus 8 receives the ON signal, the warning apparatus 8 executes thewarning to the driver by means of, for example, the voice and/or thedisplay on a monitor. When the collision preventing/collision damagereducing system receives the ON signal, the collisionpreventing/collision damage reducing system 8 executes the collisionpreventing control and/or the collision damage reducing control. Thecollision preventing/collision damage reducing system may beexemplified, for example, by an automatic steering system, a seat beltcontrol system, a seat position control system, a brake control system,and an airbag control system.

(Method for Deriving Lateral Position)

The object detecting section 6 of ECU 10 of the collision predictingapparatus 200 derives the lateral position of the obstacle in order touse the lateral position for judging the collision in the collisionjudging section 7. An explanation will be made below on the basis ofFIGS. 3 to 7 about a method for deriving the lateral position of theobstacle according to this embodiment.

As described above, the object detecting section detects the obstacle onthe basis of the target information acquired by the millimeter waveradar 1 and the target information acquired from the image picked up bythe monocular image sensor 2. When the millimeter wave radar 1 is used,it is possible to detect the relative position of the target withrespect to the subject vehicle 100. However, it is difficult for themillimeter wave radar 1 to detect the edge of the target highlyaccurately. In view of the above, in this embodiment, the edge of thetarget is detected by using the image picked up by the monocular imagesensor 2.

In particular, the edge detecting section 61 of the object detectingsection 6 is used to extract the target corresponding to the targetrecognized by the millimeter wave radar 1, i.e., the target existing atthe position of the target detected by the millimeter wave radar 1(hereinafter referred to as “millimeter wave detection position” in somecases) from the image picked up by the monocular image sensor 2.Further, the right edge and the left edge of the target are detectedfrom the image of the extracted target (the edge is hereinafter referredto as “monocular image detection edge” in some cases).

However, it is difficult to acquire any correct information (distance)in the distal direction from the image picked up by the monocular imagesensor 2. Therefore, when the left and right edges of the target aredetected from the image picked up by the monocular image sensor 2, anyedge of an object or a pattern, which actually exists distally from thetarget as viewed from the subject vehicle 100, is erroneously detectedas the edge of the target in some cases.

FIG. 3 shows a millimeter wave detection position and monocular imagedetection edges in relation to an image picked up or photographed by themonocular image sensor. FIG. 3( a) shows a case in which the edges ofthe target are detected normally, and FIG. 3( b) shows a case in whichthe edge of the target is erroneously detected. In FIGS. 3( a) and 3(b),the target is the “proximal electric pole”. As shown in FIG. 3( b), whenthe edges are detected from the image picked up by the monocular imagesensor, the edge of the “distal electric pole” is erroneously detectedas the edge of the “proximal electric pole” in some cases.

An explanation will now be made on the basis of FIG. 4 about a detectionresult obtained when the obstacle is detected by using the millimeterwave radar and a stereo image sensor unlike the embodiment of thepresent invention. Also in this case, the target, which exists at themillimeter wave detection position, is extracted from an image picked upor photographed by the stereo image sensor. The right edge and the leftedge of the target are detected from the image of the extracted target(the edges are hereinafter referred to as “stereo image detection edges”in some cases). FIG. 4 shows the time-dependent position change of themillimeter wave detection position and the stereo image detection edgewith respect to the subject vehicle when the “proximal electric pole” isthe target. In FIG. 4, the drawing, in which t=n+1 is given, depicts thesituation obtained 50 ms after the situation depicted in the drawing inwhich t=n is given. In the drawing in which t=4 is given, the blankedarrow indicates the locus of the target derived on the basis of themillimeter wave detection position and the stereo image detection edge.

It is possible to acquire correct information in the distal directionfrom the image picked up by the stereo image sensor. Therefore, when theedge is detected from the image picked up by the monocular image sensor,even if the “distal electric pole” exists distally as compared with the“proximal electric pole”, then the edge of the “distal electric pole” isnot erroneously detected as the edge of the “proximal electric pole” asshown in FIG. 4.

Therefore, the lateral position of the target center can be derivedhighly accurately on the basis of the stereo image detection edge.Therefore, as shown in the drawing in which t=4 is given, the locus ofthe target can be derived highly accurately on the basis of themillimeter wave detection positions and the stereo image detectionedges. As a result, it is also possible to highly accurately predict thefuture locus of the target.

On the other hand, an explanation will be made on the basis of FIG. 5about a detection result obtained when the obstacle is detected by usingthe millimeter wave radar and the monocular image sensor as performed inthis embodiment. FIG. 5 shows the time-dependent position change of themillimeter wave detection position and the monocular image detectionedge with respect to the subject vehicle when the “proximal electricpole” is the target. In FIG. 5, the drawing, in which t=n+1 is given,depicts the situation obtained 50 ms after the situation depicted in thedrawing in which t=n is given, in the same manner as in FIG. 4. In thedrawing in which t=5 is given, the blanked arrow indicates the locus ofthe target derived on the basis of the millimeter wave detectionposition and the monocular image detection edge.

As described above, when the edge is detected from the image picked upby the monocular image sensor, the edge of the “distal electric pole” iserroneously detected as the edge of the “proximal electric pole” in somecases. Therefore, it is difficult to highly accurately derive thelateral position of the target center on the basis of the monocularimage detection edge. Therefore, when the locus of the target is derivedon the basis of the millimeter wave detection positions and themonocular image detection edges, the locus is erroneously derived insome cases as shown in the drawing in which t=5 is given. In such asituation, it is also difficult to highly accurately predict the futurelocus of the target.

If the collision is judged between the subject vehicle 100 and thetarget in the collision judging section 7 on the basis of theerroneously predicted locus of the target, it is feared that anyerroneous judgment may be caused. Accordingly, in this embodiment, thefollowing lateral position deriving process is performed for the imagepicked up by the monocular image sensor 2 in order to derive the lateralposition of the target to be used for the collision judgment in thecollision judging section 7. FIGS. 6 and 7 show images of the lateralposition deriving process according to this embodiment.

In the lateral position deriving process according to this embodiment,as shown in FIG. 6, locus approximation lines, which are straight linesor predetermined curved lines for approximating the loci of the rightedge and the left edge respectively, are derived for the plurality ofmonocular image detection edges detected by the edge detecting section61 every time when a predetermined period of time elapses (every timewhen 50 ms elapses in this embodiment), in the locus approximation linederiving section 62 of the object detecting section 6. Any one of FIGS.6( a) and 6(b) shows the monocular image detection edges detected inFIG. 5. An alternate long and short dash line shown in FIG. 6( a)indicates the locus approximation line derived for the left edges. Analternate long and short dash line shown in FIG. 6( b) indicates thelocus approximation line derived for the right edges. A method forderiving the locus approximation line is previously determined. In thiscase, it is allowable to use any known method including, for example,the least square method and the spline interpolation method. The“predetermined curve” means the curve derived in accordance with thepreviously determined approximation method.

As shown in FIG. 6( a), it is noted that all of the left edges arenormally detected. Therefore, all of the five edges exist on the locusapproximation line. On the other hand, as shown in FIG. 6( b), theedges, which are erroneously detected, are included in the right edges.Therefore, only the three of the five edges exist on the locusapproximation line. In this procedure, the edge, which exists within apredetermined allowable range from the locus approximation line, iscounted assuming that the edge exists on the locus approximation line,even when the edge is not disposed at the position completely coincidentwith the locus approximation line. In FIGS. 6( a) and 6(b), the edgessurrounded by circles indicate the “edges existing on the locusapproximation line”.

In other words, it can be judged that the edge, which has the largernumber of edges existing on the locus approximation line, has the higherreliability as compared with the edge which has the smaller number ofedges existing on the locus approximation line. Accordingly, thereliabilities of the right edge and the left edge are calculated on thebasis of the numbers of edges existing on the locus approximation linesin the selecting section 63. The edge, which is selected from the rightedge and the left edge and which has the higher reliability (i.e., theedge having the larger number of edges existing on the locusapproximation line (left edge in FIG. 6)), is selected as the true edgeof the target.

In the lateral position deriving section 64, the lateral position of thetarget is derived on the basis of the position of the edge (hereinafterreferred to as “selected edge” in some cases) selected as the true edgeof the target in the selecting section 63. More specifically, as shownin FIG. 7, the future locus of the selected edge (left edge in FIG. 7)is firstly predicted on the basis of the past locus approximation linein the locus predicting section 641. In FIG. 7, the arrow, which isindicated by an alternate long and short dash line, represents the pastlocus approximation line and the predicted future locus of the selectededge.

Subsequently, in the collision position predicting section 642, thecollision position between the target and the subject vehicle 100 ispredicted as the position at which the distance in the front-backdirection between the selected edge and the subject vehicle 100 is zero,on the basis of the future locus of the selected edge predicted in thelocus predicting section 641 and the orbit of the subject vehicle 100calculated in the subject vehicle orbit calculating section of ECU 10.In FIG. 7, a broken line indicates the collision position.

Further, in the lateral position deriving section 64, the lateral widthWt of the target is estimated in the lateral width estimating section643. In this case, any known method is usable as the lateral widthestimating method. Specifically, it is possible to exemplify, forexample, a method in which the average value of lateral widths of thetarget derived from the monocular image detection edges is calculated asthe lateral width Wt of the target, and a method in which the lateralwidth Wt of the target is derived on the basis of the type of the targetestimated from the intensity of the received wave in the millimeter waveradar 1.

In the target center lateral position deriving section 644, the lateralposition of the target center, which is provided at the collisionposition predicted in the collision position predicting section 642, isderived. Specifically, the position, which is deviated toward the otheredge (right edge in FIG. 7) by ½ of the lateral width Wt of the targetestimated in the lateral width estimating section 643 from the positionof the selected edge at the collision position, is derived as thelateral position of the target center at the collision position. In FIG.7, the position, which is indicated by the blanked triangle, representsthe lateral position of the target center at the collision position.

According to the lateral position deriving method as described above,the edge, which is included in the right edge and the left edge of themonocular image detection edges and which has the higher reliability, isselected as the true edge of the target. Further, the lateral positionof the target center is derived at the collision position on the basisof the position of the edge having the higher reliability. Therefore,even when the monocular image sensor is used as the image sensor, it ispossible to highly accurately derive the lateral position of the targetcenter at the collision position.

In the collision judging section 7, the collision judgment is executedon the basis of the lateral position of the target center at thecollision position derived in the object detecting section 6.Accordingly, it is possible to more highly accurately judge whether ornot the subject vehicle 100 and the obstacle collide with each other.

(Collision Judgment Flow)

An explanation will be made on the basis of a flow chart shown in FIGS.8 and 9 about a flow of the collision judgment for the subject vehicleand the obstacle according to this embodiment. This flow is repeatedlyexecuted at predetermined intervals by ECU 10.

In this flow, firstly, in Step S101, the target, which exists at themillimeter wave detection position, is extracted from the image pickedup by the monocular image sensor 2.

Subsequently, in Step S102, the right edge and the left edge aredetected from the image of the target extracted in Step S101. Theprocesses in Steps S101 and S102 are executed by the edge detectingsection 61.

Subsequently, in Step S103, the locus approximation lines are derivedfor the right edge and the left edge respectively in relation to theplurality of monocular image detection edges detected in Step S102. Theprocess in Step S103 is executed by the locus approximation linederiving section 62.

Subsequently, in Step S104, the reliabilities of the right edge and theleft edge are calculated on the basis of the numbers of the edgesexisting on the locus approximation lines derived in Step S103.

Subsequently, in Step S105, the edge, which is either the right edge orthe left edge and which has the higher reliability calculated in StepS104, is selected as the true edge of the target. The processes in StepsS104 and S105 are executed by the selecting section 63.

Subsequently, in Step S106, the future locus of the selected edgeselected in Step S105 is predicted. The process in Step S106 is executedby the locus predicting section 641.

Subsequently, in Step S107, the collision position between the targetand the subject vehicle 100 is predicted on the basis of the futurelocus of the selected edge predicted in Step S106 and the orbit of thesubject vehicle 100 calculated in the subject vehicle orbit calculatingsection. The process in Step S107 is executed by the collision positionpredicting section 642.

Subsequently, in Step S108, the lateral width Wt of the target isestimated. The process in Step S108 is executed by the lateral widthestimating section 643.

Subsequently, in Step S109, the position, which is deviated toward theother edge by ½ of the lateral width Wt of the target estimated in StepS108, is derived as the lateral position of the target center at thecollision position from the position of the selected edge at thecollision position predicted in Step S107. The process in Step S109 isexecuted by the target center lateral position deriving section 644.

Subsequently, in Step S110, the collision probability Pc between thetarget and the subject vehicle 100 is calculated on the basis of thelateral position of the target center at the collision position derivedin Step S109.

Subsequently, in Step S111, it is judged whether or not the collisionprobability Pc, which is calculated in Step S110, is not less than thereference probability Pcbase. In this case, the reference probabilityPcbase is the value preset as the threshold value at which it is to bejudged that the target and the subject vehicle 100 collide with eachother.

If the affirmative judgment is made in Step S111, it is subsequentlyjudged in Step S112 that the target and the subject vehicle 100 collidewith each other. On the other hand, if the negative judgment is made inStep S111, it is subsequently judged in Step S113 that the target andthe subject vehicle 100 do not collide with each other. The processes inSteps S110 to S113 are executed by the collision judging section 7. Ifit is judged in Step S112 that the target and the subject vehicle 100collide with each other, the collision judging section 7 transmits theON signal to the warning apparatus 8 and the collisionpreventing/collision damage reducing system 9.

(Relationship Between Constitutive Elements of this Embodiment andConstitutive Requirements of the Present Invention)

In this embodiment, the object detecting section 6 corresponds to theobject detecting apparatus according to the present invention. Therelationship between the constitutive elements of the object detectingsection 6 according to this embodiment and the constitutive requirementsof the present invention is as follows. The edge detecting section 61corresponds to the edge acquiring means according to the presentinvention. The locus approximation line deriving section 62 correspondsto the locus approximation line deriving means according to the presentinvention. The selecting section 63 corresponds to the selecting meansaccording to the present invention. The lateral position derivingsection 64 corresponds to the lateral position deriving means accordingto the present invention. The locus predicting section 641 correspondsto the locus predicting means according to the present invention. Thecollision position predicting section 642 corresponds to the collisionposition predicting means according to the present invention. Thelateral width estimating section 643 corresponds to the lateral widthestimating means according to the present invention.

In this embodiment, Steps S101 and S102 in the flow chart shown in FIG.8 correspond to the edge acquiring step according to the presentinvention. Step S103 in the flow chart shown in FIG. 8 corresponds tothe locus approximation line deriving step according to the presentinvention. Steps S104 and S105 in the flow chart shown in FIG. 8correspond to the selecting step according to the present invention.Steps S106 to S109 in the flow chart shown in FIG. 8 correspond to thelateral position deriving step according to the present invention.

Second Embodiment

A first embodiment of the present invention will be explained on thebasis of FIGS. 10 to 14. In this section, an explanation will be madeabout only the points or features different from those of the firstembodiment.

(Schematic Arrangement)

FIG. 10 shows a block diagram illustrating a schematic arrangement of anobject detecting section 6 according to this embodiment. As shown inFIG. 10, the object detecting section 6 according to this embodimentincludes a weight applying section 65 and a reliability total valuecalculating section 66 in addition to the edge detecting section 61, thelocus approximation line deriving section 62, the selecting section 63,and the lateral position deriving section 64. Details of the weightapplying section 65 and the reliability total value calculating section66 will be described later on.

(Method for Deriving Lateral Position)

A method for deriving the lateral position of an obstacle according tothis embodiment will be explained on the basis of FIGS. 11 to 13. FIG.11 shows a detection result obtained when the obstacle is detected byusing a millimeter wave radar and a monocular image sensor according tothis embodiment. FIG. 11 shows the time-dependent position change of themillimeter wave detection position and the monocular image detectionedge with respect to the subject vehicle when a “proximal electric pole”is the target in the same manner as in FIG. 5. In FIG. 11, the drawing,in which t=n+1 is given, depicts the situation obtained 50 ms after thesituation depicted in the drawing in which t=n is given in the samemanner as in FIG. 5.

In the case of FIG. 11, a “traversing vehicle” exists distally ascompared with the “proximal electric pole”. In this situation, if theedge is detected from an image picked up or photographed by themonocular image sensor, the edge of the “traversing vehicle” iserroneously detected as the edge of the “proximal electric pole” in somecases. In FIG. 11, the right edge is erroneously detected every time.The erroneous detection as described above may be also caused when afixed object such as the “distal electric pole” exists distally ascompared with the “proximal electric pole” as the target, as caused inthe case shown in FIG. 5. However, the erroneous detection tends to becaused more often when the object, which exists distally as comparedwith the target, is a moving object such as the “traversing vehicle”which moves in the transverse direction or the lateral direction.

In this embodiment, the following lateral position deriving process isperformed for the image picked up by the monocular image sensor 2 in theobject detecting section 6 in order to derive the lateral position ofthe target to be used for the collision judgment in the collisionjudging section 7. FIGS. 12 and 13 show images of the lateral positionderiving process according to this embodiment.

Also in the lateral position deriving process according to thisembodiment, as shown in FIG. 12, locus approximation lines, which arestraight lines or predetermined curved lines for approximating the lociof the right edge and the left edge respectively, are derived for theplurality of monocular image detection edges detected by the edgedetecting section 61 every time when a predetermined period of timeelapses (every time when 50 ms elapses in this embodiment), in the locusapproximation line deriving section 62 of the object detecting section6. Any one of FIGS. 12( a) and 12(b) shows the millimeter wave detectionpositions and the monocular image detection edges detected in FIG. 11.An alternate long and short dash line shown in FIG. 12( a) indicates thelocus approximation line derived for the left edges. An alternate longand short dash line shown in FIG. 12( b) indicates the locusapproximation line derived for the right edges. A method for derivingthe locus approximation line is the same as or equivalent to that in thefirst embodiment.

If the right edge is erroneously detected every time, as shown in FIG.12( b), all of the five edges exist on the locus approximation line evenin the case of the erroneous detection, in the same manner as the leftedge normally detected as shown in FIG. 12( a). Also in this embodiment,the edge, which exists within a predetermined allowable range from thelocus approximation line even when the edge is not disposed at theposition completely coincident with the locus approximation line, iscounted assuming that the edge exists on the locus approximation line inthe same manner as the case of the first embodiment. In FIGS. 12( a) and12(b), the edges, which are surrounded by circles, indicate the “edgesexisting on the locus approximation line”.

Therefore, when the reliability of the edge is calculated on the basisof the number of edges existing on the locus approximation line, thereliability of the edge detected normally every time and the reliabilityof the edge detected erroneously every time are equivalent to oneanother. In such a situation, it is difficult to select the true edge ofthe target by means of the lateral position deriving process accordingto the first embodiment.

In view of the above, in the lateral position deriving process accordingto this embodiment, the weight is applied to the reliability in theweight applying section 65 on the basis of the distance from themillimeter wave detection position for each of the right edges and theleft edges detected by the edge detecting section 61. In this procedure,the edge, which is disposed far from the millimeter wave detectionposition, has such a high possibility that the edge may be erroneouslydetected as compared with the edge which is disposed near to themillimeter wave detection position. Therefore, the weight applyingsection 65 applies the weight to the right edge and the left edge sothat the edge, which is disposed nearer to the millimeter wave detectionposition, has the higher reliability.

Further, the reliability total value calculating section 66 calculatesthe total values of the reliabilities in relation to the both edges suchthat the plurality of reliabilities, to which the weights are applied bythe weight applying section 65, are totalized for the right edges andthe left edges respectively.

For example, in the case of the situation shown in FIG. 12, the leftedges are disposed nearer to the millimeter wave detection position ascompared with the right edges in relation to all of the five. In thisprocedure, it is assumed that the weight, which is applied to thosedisposed nearer to the millimeter wave detection position, is 1.1 pointsand the weight, which is applied to those disposed farther from themillimeter wave detection position, is 0.9 point in relation to theright edges and the left edges. On this assumption, the total value ofthe reliabilities of the left edges is 5.5 points, and the total valueof the reliabilities of the right edges is 4.5 points.

In the selecting section 63, the edge (left edge in FIG. 12), which isincluded in the right edge and the left edge and which has the largertotal value of the reliabilities calculated in the reliability totalvalue calculating section 66, is selected as the true edge of thetarget. Accordingly, even if one edge is erroneously detected every timeas shown in FIG. 11, the edge, which has the higher reliability, can beselected as the true edge of the target from the right edge and the leftedge in relation to the monocular image detection edges.

In this embodiment, the lateral position of the target is derived inaccordance with the same or equivalent method as that of the firstembodiment on the basis of the position of the selected edge selected asdescribed above in the lateral position deriving section 64. That is, asshown in FIG. 13, the future locus of the selected edge is predicted inthe locus predicting section 641. Subsequently, the collision positionbetween the target and the subject vehicle 100 is predicted in thecollision position predicting section 642. Further, the lateral width Wtof the target is estimated in the lateral width estimating section 643.The lateral position of the target center at the collision position isderived in the target center lateral position deriving section 644. InFIG. 13, the arrow, which is indicated by an alternate long and shortdash line, represents the past locus approximation line and thepredicted future locus of the selected edge, a broken line representsthe collision position, and the position indicated by a blanked trianglerepresents the lateral position of the target center at the collisionposition in the same manner as in FIG. 7.

The lateral position of the target center at the collision position isalso derived on the basis of the position of the edge which has thehigher reliability and which is selected from the right edge and theleft edge of the monocular image detection edges in the same manner asin the first embodiment by means of the lateral position deriving methodas described above. Therefore, even when the monocular image sensor isused as the image sensor, it is possible to highly accurately derive thelateral position of the target center at the collision position.

(Collision Judgment Flow)

An explanation will be made on the basis of a flow chart shown in FIG.14 about a flow of the collision judgment for the subject vehicle andthe obstacle according to this embodiment. This flow is repeatedlyexecuted at predetermined intervals by ECU 10. In this flow, Steps S201to S203 are added to the flow chart shown in FIG. 8. Therefore, onlySteps S201 to S203 will be explained, and the other steps are omittedfrom the explanation.

In this flow, the process of Step S201 is executed next to Step S104. InStep S201, it is judged whether or not the reliability of the right edgeand the reliability of the left edge, which are calculated in Step S104,are equivalent to one another. In this procedure, if the number of edgesexisting on the locus approximation line is identical between the rightedge and the left edge, it is judged that the reliabilities of the bothedges are equivalent to one another. If the negative judgment is made inStep S201, the process of Step S105 is subsequently executed. In thisprocedure, the processes to be executed thereafter are the same as thosein the first embodiment. On the other hand, if the affirmative judgmentis made in Step S201, the process of Step S202 is subsequently executed.

In Step S202, the reliability weight application is executed withrespect to the right edge and the left edge in relation to the pluralityof monocular image detection edges detected in Step S102. In thisprocedure, as described above, the weight is applied so that the edge,which is disposed nearer to the millimeter wave detection position, hasthe higher reliability. The process of Step S201 is executed by theweight applying section 65.

Subsequently, in Step S203, the plurality of reliabilities of rightedges and left edges subjected to the weight application in Step S202are totalized respectively to calculate the total values of thereliabilities in relation to the both edges. The process of Step S203 isexecuted by the reliability total value calculating section 66.

Subsequently, in Step S105, the true edge of the target is selected. Inthis procedure, in Step S105, the edge, which is included in the rightedge and the left edge and which has the higher total value of thereliabilities calculated in Step S203, is selected as the true edge ofthe target. The processes to be performed thereafter are the same as orequivalent to those performed in the first embodiment.

(Relationship Between Constitutive Elements of this Embodiment andConstitutive Requirements of the Present Invention)

In this embodiment, the weight applying section 65 corresponds to theweight applying means according to the present invention, and thereliability total value calculating section 66 corresponds to thereliability total value calculating means according to the presentinvention.

In this embodiment, Step S202 in the flow chart shown in FIG. 14corresponds to the weight applying step according to the presentinvention, and Step S203 in the flow chart corresponds to thereliability total value calculating step according to the presentinvention.

DESCRIPTION OF THE REFERENCE SIGNS

-   1: millimeter wave radar-   2: monocular image sensor-   3: steering angle sensor-   4: yaw rate sensor-   5: wheel pulse sensor-   6: object detecting section-   7: collision judging section-   8: warning apparatus-   9: collision preventing/collision damage reducing system-   10: ECU-   61: edge detecting section-   62: locus approximation line deriving section-   63: selecting section-   64: lateral position deriving section-   65: weight applying section-   66: reliability total value calculating section-   100: vehicle-   200: collision predicting apparatus-   641: locus predicting section-   642: collision position predicting section-   643: lateral width estimating section-   644: target center lateral position deriving section.

1-6. (canceled)
 7. An object detecting apparatus for detecting an objecton the basis of target information acquired by a radar and targetinformation acquired from an image picked up by a monocular imagesensor, the object detecting apparatus comprising: an edge acquiringmeans which extracts a target corresponding to a target recognized bythe radar, from the image picked up by the monocular image sensor andwhich acquires a right edge and a left edge of the extracted target; alocus approximation line deriving means which derives, for the bothedges, locus approximation lines as straight lines or predeterminedcurved lines for approximating loci of the right edge and the left edgeacquired by the edge acquiring means; a selecting means which selects,as a true edge of the target, the edge having a larger number of edgesexisting on the locus approximation line, from the right edge and theleft edge acquired by the edge acquiring means; and a lateral positionderiving means which derives a lateral position of the target on thebasis of a position of the edge selected as the true edge by theselecting means.
 8. The object detecting apparatus according to claim 7,further comprising: a weight applying means which applies reliabilityweights to the right edge and the left edge acquired by the edgeacquiring means so that the edge, which is disposed nearer to a positionof the target recognized by the radar, has a higher reliability; and areliability total value calculating means which totalizes a plurality ofreliabilities having the weights applied by the weight applying meansfor each of the right edge and the left edge to thereby calculate totalvalues of the reliabilities for the both edges, wherein: the selectingmeans selects, as the true edge of the target, the edge which has thelarger total value of the reliabilities calculated by the reliabilitytotal value calculating means, from the right edge and the left edge, ifthe number of the edges existing on the locus approximation line isidentical between the right edge and the left edge acquired by the edgeacquiring means.
 9. The object detecting apparatus according to claim 7,wherein the lateral position deriving means includes: a locus predictingmeans which predicts a future locus of the edge selected as the trueedge by the selecting means; and a collision position predicting meanswhich predicts a collision position between the target and a vehicle asa position at which a distance between the edge and the vehicle in thefront-back direction is zero on the basis of the future locus of theedge predicted by the locus predicting means, wherein: a lateralposition of a center in a lateral direction of the target, which is tobe provided at the collision position, is derived on the basis of aposition at the collision position of the edge selected as the true edgeby the selecting means.
 10. The object detecting apparatus according toclaim 8, wherein the lateral position deriving means includes: a locuspredicting means which predicts a future locus of the edge selected asthe true edge by the selecting means; and a collision positionpredicting means which predicts a collision position between the targetand a vehicle as a position at which a distance between the edge and thevehicle in the front-back direction is zero on the basis of the futurelocus of the edge predicted by the locus predicting means, wherein: alateral position of a center in a lateral direction of the target, whichis to be provided at the collision position, is derived on the basis ofa position at the collision position of the edge selected as the trueedge by the selecting means.
 11. The object detecting apparatusaccording to claim 7, wherein: the lateral position deriving meansincludes a lateral width estimating means which estimates a lateralwidth of the target; and a position, which is deviated toward the otheredge by ½ of the lateral width of the target estimated by the lateralwidth estimating means from the position of the edge selected as thetrue edge by the selecting means, is derived as a lateral position of acenter in a lateral direction of the target.
 12. The object detectingapparatus according to claim 8, wherein: the lateral position derivingmeans includes a lateral width estimating means which estimates alateral width of the target; and a position, which is deviated towardthe other edge by ½ of the lateral width of the target estimated by thelateral width estimating means from the position of the edge selected asthe true edge by the selecting means, is derived as a lateral positionof a center in a lateral direction of the target.
 13. An objectdetecting method for detecting an object on the basis of targetinformation acquired by a radar and target information acquired from animage picked up by a monocular image sensor, the object detecting methodcomprising: an edge acquiring step of extracting a target correspondingto a target recognized by the radar, from the image picked up by themonocular image sensor and acquiring a right edge and a left edge of theextracted target; a locus approximation line deriving step of deriving,for the both edges, locus approximation lines as straight lines orpredetermined curved lines for approximating loci of the right edge andthe left edge acquired in the edge acquiring step; a selecting step ofselecting, as a true edge of the target, the edge having a larger numberof edges existing on the locus approximation line, from the right edgeand the left edge acquired in the edge acquiring step; and a lateralposition deriving step of deriving a lateral position of the target onthe basis of a position of the edge selected as the true edge in theselecting step.
 14. The object detecting method according to claim 13,further comprising: a weight applying step of applying reliabilityweights to the right edge and the left edge acquired in the edgeacquiring step so that the edge, which is disposed nearer to a positionof the target recognized by the radar, has a higher reliability; and areliability total value calculating step of totalizing a plurality ofreliabilities having the weights applied in the weight applying step foreach of the right edge and the left edge to thereby calculate totalvalues of the reliabilities for the both edges, wherein: the edge, whichhas the larger total value of the reliabilities calculated in thereliability total value calculating step, is selected as the true edgeof the target from the right edge and the left edge in the selectingstep, if the number of the edges existing on the locus approximationline is identical between the right edge and the left edge acquired inthe edge acquiring step.